Skip to content

fix(sync): (Codex) Sync digest consistency / eventual consistency#1393

Merged
CodFrm merged 2 commits into
mainfrom
fix/sync/008
May 6, 2026
Merged

fix(sync): (Codex) Sync digest consistency / eventual consistency#1393
CodFrm merged 2 commits into
mainfrom
fix/sync/008

Conversation

@cyfung1031
Copy link
Copy Markdown
Collaborator

@cyfung1031 cyfung1031 commented May 1, 2026

同步流程不再完全依赖“上传后立刻重新 list() 云端文件”来刷新 file_digest。现在 pushScript() 会在脚本和 .meta.json 成功写入后，返回这两个文件的内容摘要；syncOnce() 会收集所有成功上传的摘要，并在 updateFileDigest() 里把它们合并进云端列表结果。即使 Google Drive / S3 这类存储刚上传后目录列表短暂漏掉新文件，本地 digest 也会保留刚刚确认写入的文件，避免下次同步误判为缺失或变更后重复上传。

也同步处理了安装脚本触发的单次上传路径：scriptInstall() 现在同样会把 pushScript() 返回的摘要传给 updateFileDigest()。

@cyfung1031 cyfung1031 changed the title fix(sync): Sync digest consistency / eventual consistency fix(sync): (Codex) Sync digest consistency / eventual consistency May 1, 2026
@cyfung1031 cyfung1031 added the CloudSync Related to CloudSync label May 1, 2026
@CodFrm
Copy link
Copy Markdown
Member

CodFrm commented May 6, 2026

Code review

Found 1 issue:

  1. After this PR, the digest store is populated with locally-computed md5OfText(content) for files that were just pushed. On the next sync, line 436 compares that stored MD5 against file.script!.digest returned by fs.list(). For most filesystem backends (webdav.ts:118 returns item.etag, onedrive.ts:178 returns val.eTag, dropbox.ts:204 returns item.content_hash, s3.ts:239 returns the S3 ETag, zip.ts:56 returns ""), FileInfo.digest is the provider-native ETag/content-hash — not an MD5. The equality check therefore never matches, so every previously-pushed script is detected as "changed" on every subsequent sync and re-uploaded. This regresses the optimization on roughly 5 of 7 backends; only Google Drive (md5Checksum) and Baidu (md5) happen to match.

Concretely, in updateFileDigest, Object.assign(newFileDigestMap, knownFileDigestMap) overwrites the cloud-native digest from fs.list() with the local MD5; line 436 then compares incompatible formats on the next pass.

scriptcat/src/app/service/service_worker/synchronize.ts

Lines 432 to 440 in 68f3606

}
})()
);
return;
}
// 过滤掉无变动的文件
if (fileDigestMap[file.script!.name] === file.script!.digest) {
return;
}

scriptcat/src/app/service/service_worker/synchronize.ts

Lines 549 to 560 in 68f3606

async updateFileDigest(fs: FileSystem, knownFileDigestMap: FileDigestMap = {}) {
const newList = await fs.list();
const newFileDigestMap: FileDigestMap = {};
for (const file of newList) {
newFileDigestMap[file.name] = file.digest;
}
Object.assign(newFileDigestMap, knownFileDigestMap);
await this.storage.set("file_digest", newFileDigestMap);
return;
}
// 删除云端脚本数据

scriptcat/src/app/service/service_worker/synchronize.ts

Lines 617 to 625 in 68f3606

[filename]: md5OfText(scriptCode),
[metaFilename]: md5OfText(metaJson),
};
} catch (e) {
logger.error("push script error", Logger.E(e));
throw e;
}
}

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@CodFrm
fix(sync): preserve cloud-native digest in updateFileDigest
1b4e05c 
云端 list 已能返回原生 digest 时不应被本地 md5 覆盖。webdav/onedrive/s3
返回 etag、dropbox 返回 content_hash，与 md5OfText 格式不一致，强制覆盖
会让下次同步比对必失败，导致未变动脚本被反复识别为已变动。改为只在云端
列表暂时漏掉刚上传的文件时用本地 md5 兜底。
@CodFrm CodFrm merged commit 074be01 into main May 6, 2026
4 checks passed
@CodFrm CodFrm deleted the fix/sync/008 branch May 6, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CloudSync Related to CloudSync

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@cyfung1031 @CodFrm